Multi-point motion picture encoding and decoding apparatus

ABSTRACT

A multi-point video conference system includes a plurality of terminals and a network. Each terminal includes a coder to code and to packetize at least a motion picture in a block unit and a multi-point video decoder to sequentially receive and to decode packets sent from decoders at other points. The decoder includes a video processor to process video information in a time sharing manner, a channel selection controller to decide a transmission source of a received packet according to information of the packet and to notify the source to the video processor, a shared frame memory to store therein video information from the plural terminals, and a display frame memory to output therefrom the video information in a synchronous fashion.

This application is a 37 CFR §1.60 continuation application of priorapplication Ser. No. 08/440,710, filed May 15, 1995, now U.S. Pat. No.5,568,184, which was a continuation application of prior applicationSer. No. 08/170,750 filed Dec. 21, 1993 (issued Aug. 29, 1995, as U.S.Pat. No. 5,446,491).

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a multi-point motion picture conferencesystem in which motion pictures are communicated via communication linesbetween terminals respectively installed at remote points, and inparticular, to a motion picture coding (encoding) and decoding apparatusto send coded motion picture information to terminals at multiple pointsand to receive coded motion picture information sent therefrom.

2. Description of the Related Art

FIG. 1 shows a first example of a multi-point video conference system ofa conventional technology. According to features of the example, therecan be used terminals for use in a one-to-one (terminal-to-terminal)motion picture communication and there is disposed a multi-pointconference connecting device 106 to simultaneously establish connectionsfor motion pictures sent from the terminals installed at many points.For example, a terminal 101a disposed at a point A for a one-to-onemotion picture or video communication includes an imaging device or avideo camera 102a to produce an image of an object, a coder or anencoder 103a to encode a motion picture in conformity with acharacteristic of a communication line and to send the picture via theline, a decoder 104a to decode the encoded information thus received soas to restore the original image, and a display 105a to present thereonthe restored image. The device 106 includes coders 109a to 109d anddecoders 108a to 108d respectively as many as there are terminalsdisposed as above and a video (image) synthesizer 107 to establishconnections between the coders 109a to 109d and the decoders 108a to108d. Each of the decoders 108a to 108d associated with respectivecommunication channels decodes codes or symbols sent via the channel torestore the image. The decoded image thus prepared is subjected to asynthesizing process in the synthesizer 107 and then is again encoded bythe coders 109a to 109d. The coded image is thereafter transmitted tothe related terminals.

FIG. 2 shows a second example of the conventional multi-point videoconference system. In accordance with an aspect of the example, each ofthe terminals includes a plurality of decoders respectivelycorresponding to terminals which may possibly be specified ascommunication partners. With this provision, the connecting devicededicated to the conference system of FIG. 1 can be dispensed with suchthat coded information sent via each channel can be independentlycontrolled. For example, a terminal 201a at a point A includes a videocamera 202a to produce an image of an object, a coder 203a to encodedata of a motion picture in conformity with a characteristic of acommunication line and to send the encoded data via the line, decoders204ab, 204ac, and 204ad to receive codes via related channels and tothereby restore the data of the respective motion pictures, and adisplay 205a to display the restored image on a screen thereof. Theconstituent elements conduct operations similar to those of theassociated components of FIG. 1. However, this example is different fromthe preceding example in that the codes received via the respectivechannels are decoded by the decoders 204ab, 204ac, and 204ad in anindependent manner, thereby displaying synthesized images.

FIG. 3 shows a third example of the multi-point video conference systemof the prior art. This system has been described in the JP-A-63-276938.It is assumed that this example is configured to be operated with apacket switching network. In this system, according to the number ofpoints connected to each other via the network, an image attained from avideo camera is encoded through contraction or compression thereof andis then transmitted to a terminal. As for the configuration, descriptionwill be representatively given of a terminal 301a arranged at a point A.The terminal 301a includes a video camera 302a, a screen contractiondevice 303a to minimize the size of image data 309 inputted thereto, acoder 304a to encode the screen image data 309 thus compressed and tothereby produce a packet output 310a, a multiplexer 305a to receivepackets 310b to 310e sent from related terminals and to producemultiplexed data in the frame unit, a decoder 306a to decode the data soas to produce a synthesized image 312 in the frame unit, and a display307a to display the synthesized image thus decoded.

In a case where the terminal A 310a is connected to other terminalsdisposed respectively at the remaining four points, the the size of theinput image supplied from the imaging device 302a is compressedaccording to a contraction scale "1/4". Namely, the size is minimized tobe a quarter of the original size, thereby producing the contractedimage 309. The resultant image is encoded by the coder 304a. On thereceiver side, encoded data items respectively sent from the fourterminals at the respective points are decoded for each frame in a timesharing manner so as to produce the synthesized image 312.

The conventional technologies are respectively attended with problems asfollows.

In the first example, there is required the connecting device dedicatedto the conference system. The number of connectible terminals is limitedby that of the encoder-decoder pairs of the system. Moreover, with theconnecting device, only one conference can be held at a time. Inaddition, due to repetitious encoding and decoding operations, the imagequality is considerably deteriorated.

The second example necessitates decoders as many as there are channelswhich may possibly be used to receive data. The number of connectibleterminals is limited by that of the decoders. Furthermore, an equalnumber of terminals and decoders are required to be disposed.Consequently, for example, when the number of connected terminals isincreased, the number of decoders is also increased. This remarkablyincreases the size of the terminal and hence the terminal resultantlybecomes to be expensive.

According to the third example, the size of the presentable image isrestricted by the number of connected terminals. In addition, althoughthe image processing is accomplished in a frame unit, it is notparticularly disclosed how to suppress mismatching with respect to theframe speed.

Moreover, each of the technologies above is attended with a problem thatthe number of connection channels is methodologically limited.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide amulti-point video conference system in which video communications to andfrom a plurality of points can be implemented without using anyconnecting device dedicated to the communications and in which eachterminal includes a decoder so that pictures or images from a pluralityof points can be concurrently received and displayed.

Another object of the present invention is to provide a multi-pointvideo conference system in which the communication channels between therespective terminals are independent of each other such that an imagereceiving channel can be added to or deleted from each terminal withinthe processing capacity of the decoder thereof.

According to an aspect of the present invention, each terminal connectedto a packet network includes a multi-point motion picture decoderincluding means for sequentially receiving codes of motion pictures inthe form of packets via a multiplicity of channels and processing thecodes from the respective channels in a time sharing manner, a sharedframe memory for storing therein decoded images from the channels, anddisplay frame memories. The number of the frame memories is associatedwith that of channels for which signals asynchronously arriving thereatare synchronized with a synchronizing signal on a signal receiving side.The terminal further includes a video coder for producing encoded videoor picture signals conforming to the decoder.

In the means of processing the codes of the respective channels in thetime sharing fashion, the codes of the received packets are processed ina receiving order thereof. Consequently, in a case where packets arereceived in an arbitrary order from the respective channels, the codesof the channels are decoded in the time sharing manner for each packet.In the shared frame memory, there can be stored the decoded images ofall channels from which the images are being received. In consequence,for any channels, the codes can be decoded according to correlationsbetween frames. Moreover, it is possible to extract from a packet achannel number indicating a source terminal of the packet so as toidentify a location in the frame memory where the decoded image of thepertinent channel is stored. With provision of the display frame memoryfor each channel, the images asynchronously received from the associatedchannels can be synchronized to be outputted therefrom without anyconflict.

In addition, regarding the system as a motion picture conference system,the number of connectible channels depends only on the code processingcapacity of the own terminal, the frame memory capacity, and thetransmission speed of the communication network. In consequence, thenumber is not methodologically influenced, i.e., the number is notrelated to the method of the system. Selection of an image or a picturefor reception thereof can be independently decided for each terminal onthe receiving side. Consequently, it is possible for the user of theterminal to participate in two or more conferences. Moreover, there isprovided a high degree of freedom of resolution for images to becommunicated between terminals. For example, while limiting the numberof connected terminals, there may be communicated high-resolutionpictures. Conversely, low-resolution pictures can be communicated to andfrom a larger number of terminals. Namely, the picture resolution can bedynamically altered in the communication between the terminals.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects and advantages of the present invention willbecome apparent by reference to the following description andaccompanying drawings wherein:

FIG. 1 is a diagram schematically showing a first conventional exampleof a (multi-point video) conference system in which a plurality ofterminal points are connected to each other via video communications;

FIG. 2 is a diagram showing a second example of the conference system ofthe prior art;

FIG. 3 is a diagram showing a third conventional example of theconference system;

FIG. 4 is a diagram showing the configuration of a multi-point videoconference system employing a multi-point video coding and decodingapparatus according to the present invention;

FIG. 5 is a diagram showing the structure of a coder of a terminal in amulti-point conference system according to the present invention;

FIG. 6 is a diagram showing the constitution of a multi-point videoencoding and decoding apparatus of a terminal in a multi-pointconference system according to the present invention;

FIG. 7 is a diagram showing the construction of a code processor toprocess received codes in a multi-point video decoder according to thepresent invention;

FIG. 8 is a diagram showing a video processor of a multi-point videodecoder according to the present invention;

FIG. 9 is a diagram showing a display frame memory section of amulti-point video decoder according to the present invention;

FIG. 10 is a signal timing chart showing operation of the display framememory according to the present invention; and

FIG. 11 is a diagram showing an alternative construction of a codeprocessor to process received codes in a multi-point video decoderaccording to the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the accompanying drawings, description will be given ofan embodiment according to the present invention.

This embodiment is implemented according to an algorithm described inthe recommendation H. 261 for video encoding method standards of theInternational Telecommunication Union Telecommunication StandardizationSector (ITU-TS or ITU-T). In this configuration, there are includedmodifications to cope with packet transmission and connections betweenmultiple points. Consequently, even when there is employed a videocoding operation based on an algorithm other than that recommended bythe ITU-TS, the advantageous effect of the present invention can be alsoattained only by conducting image processing similar to that of thepresent invention. However, the algorithm to be applied to this case isrequired to satisfy the following conditions. A screen is subdividedinto quite small partitions, namely, blocks and the video coding isachieved in the block unit; moreover, the algorithm is establishedaccording to correlation between frames. In this connection, a term"block" in the following example is substantially comparable to"macro-block" of the recommendation H. 261.

FIG. 4 shows an example of the constitution of a multi-point videoconference system according to the present invention. Although theexample includes terminals disposed at four sites, the number ofterminal points can be set to an arbitrary value n (n is an integer notless than two). Reference numerals 401a to 401d denote motion pictureconference terminals at points A to D, respectively. A reference numeral406 denotes a packet network to switch coded video packets transmittedfrom the respective terminals. Numerals 407a to 407d designate codedvideo packets transmitted from the terminals at the points A to D,respectively.

Since each terminal conducts basically the same operation, descriptionwill be representatively given of the terminal 401a at the site A. Onthe sender side, a motion picture is produced by a video camera 402asuch that resultant data of the motion picture is sent to a coder 403a.In the coder 403a, a redundant portion is removed from the data and theobtained data is subjected to processing so as to establish matchingbetween the data and the network 406 (information compression). Inaddition, information thus compressed is subdivided into small unitscalled packets 407a or cells to be sequentially sent to the network 406.On the other hand, on the receiver side, the packets 407b to 407dtransmitted respectively from the terminals 401b to 401d at therespective points are received by a multi-point video decoder 404a so asto rebuild the original motion pictures, thereby presenting the pictureson a display 405a.

As for a remarkable aspect of the present invention, description will begiven later of a specific configuration and concrete operations of thedecoder 404a.

FIG. 5 shows an example of the internal structure of the coder 403a. Asdescribed above, this embodiment is formed in conformity with thestandardization recommendation H. 261 of the CCITT. In the system, aportion 500 enclosed with dotted lines functions as a coder implementedaccording to the CCITT recommendation H. 261. The coder 500 accomplishesoperation comparable to that stipulated by the CCITT. The algorithm ofthis section may be replaced with another algorithm similar thereto.Description will now be given of operation of the coder 403a accordingto the present invention.

A video data input 550 is subdivided into blocks. According toinformation of the blocks and an image encoded at a time in advance byone frame relative to the data 550 and stored in a frame memory 502,there is obtained a motion 552 by a motion estimation processor 501.Based on the quantity of motion 552, it is decided by a block attributedeciding section 503 whether the pertinent block is subjected to anintra-frame encoding or a between-frame encoding, thereby outputting aflag 553 for changing over between these encoding operations. When thereexist a large correlation between frames, the input 550 is selected by aselector 505. For a large correlation between frames, a differencebetween the input image 550 and a motion compensation image is attainedby a subtracter 506, the compensation image being obtained throughcompensation by a motion compensation processor 504 for a motion of theimage previously encoded and stored in the frame memory 502. Thedifference is selected by the selector 505. Moreover, in the section503, there is also determined a quantization parameter 554 indicating aquantization step for a quantizer 508 according to the above informationas well as information of the line status and the like supplied from anexternal device. The parameter 554 and an address or coordinates of ablock of the quantization parameter 554 are outputted as a blockattribute 557. Next, the signal selected by the selector 505 isconverted by an orthogonal transform processor 507 into orthogonaltransform coefficients. The coefficients are then quantized by thequantizer 508 according to the parameter 554 so as to producequantization coefficients 555.

An integral number of blocks containing the coefficients 555 and theblock attribute 557 which have been converted into the variable lengthcodes as above are inputted to a packetizer section 511. A headercreated by a header generator 510 is added thereto to produce an outputsignal of an encoded video packet 556 having substantially a fixedlength. The header includes information items related to a terminal suchas a packet length, a packet number, a terminal identifier, and aterminal resolution.

Supplied to an inverse quantizer 512 are the coefficients 555 to besubjected to an inverse quantization according to the parameter 554. Inan inverse orthogonal transform section 513, the coefficients thusinversely quantized undergo an inverse orthogonal transformation. Whenthe flag 553 in the block being processed indicates the intra-frameencoding, a selector 514 selects information directly from the section513. When the flag 553 designates the between-frame encoding, thebetween-frame estimation signal and the information from the section 513are added to each other by an adder 515 to select the resultant signalas an output therefrom. The selected video signal is stored in thememory 502. Thereafter, the signal is referenced when a subsequent frameis encoded.

Referring now to FIGS. 6 to 10, description will be given of an exampleof a specific configuration of the decoder 404a. The example isconstructed also according to the recommendation H 261 of the CCITT. Inthis constitution, the algorithm may be altered in association with eachsender side.

The decoder 404a of the present invention primarily includes threecomponent sections including a code processor 411 for interpreting acoded video packet 451 inputted thereto, a video processor section 413for restoring an image according to the interpreted information, and adisplay frame memory section 415 for simultaneously producing aplurality of images thus restored.

The section 411 includes a separator 601 for splitting an encoded videopacket 651 inputted thereto into a header portion 652 and a remainingportion other than the header, a packet buffer 602 for temporarilystoring therein information of codes other than the header, a channelselection controller 603 for generating a receiving channel change-oversignal 657 according to the header 652, a decoder 604 for quantizing thecode information of the packet other than the header together withattribute information in a block-by-block fashion, and a workingregister 605 operative in relation to a change-over operation betweenchannels for temporarily storing therein from the decoder 604information related to the block being decoded.

Next, operations of primary sections of the processor 411 will bedescribed by reference to FIG. 7. The packets 451 are sequentiallytransmitted via the network 406. In the separator 601, the packet 451 ispartitioned into the header 652 and the remaining code information. Thedecoder 604 is a section to interpret code information of the codes ofthe packets other than the header, the packets being transmitted viaseveral channels in the time-sharing manner. A variable-length decoder701 receives as an input thereto the code information 751 other than theheader to decode the information 701 by reference to a code table 702.

The decoder 701 produces two kinds of information. The first kind ofinformation is block attribute information indicating a block attributesuch as information denoting either one of a block position, a motioncompensation vector, and an intra-frame or between-frame encoding. Theinformation is outputted to a block attribute first-in first-out (FIFO)memory 703 which is a constituent element of the decoder 604. As thesecond kind of information, there are produced quantizationcoefficients. The coefficients are attained by achieving an orthogonaltransformation for video or image information of a block inputted to thedecoder 701. The resultant coefficients are supplied to the videoprocessor 413.

To process in a time sharing fashion the packets transmitted through therespective channels, a plurality of working data areas 705a to 705c aredisposed in the register 605. These areas are arranged respectively inassociation with the channels. When the interpretation process isfinished for a packet in the register 605, the receiving channelinformation 657 designating a transmission source of a subsequent packetis inputted to the register 605. In accordance with the information 657,a change-over occurs between the areas 705a, 705b, and 705c. Even whenthe channel is thus switched by the mechanism, data items received fromthe channels can be independently accomplished. Namely, the decodingoperations of the respective channels are carried out without causingany interference with each other.

Returning now to FIG. 6, an example of the specific configuration of thevideo processor 413 will be described. In this connection, the videocoder 404 includes a code processor 411 for interpreting the videopacket input 451, a video processor 413 for restoring an image accordingto the interpreted information, and a display frame memory 415 forsimultaneously outputting therefrom a plurality of images. The processor411 includes a channel selection controller 603 for selecting areceiving channel. The processor 413 includes a shared frame memory 614for storing therein the plural images. Concrete configurations andoperations thereof will be described later in detail.

Next, description will be given of a connecting section between theprocessors 411 and 413. Quantization coefficients are stored in a blockdata buffer 606. The processor 413 executes processing for each block,namely, in the block unit. Consequently, the processing is commencedonly when at least one block of quantization coefficients is accumulatedin the buffer 606. When the block is thus stored, the buffer 606generates a one-block data ready flag 752. In response thereto, acontroller 704 sends a processing start signal to the video processor413. Specifically, the controller 704 outputs control signals to therespective constituent elements of the processor 413. However, only someof the control signals are shown in the drawings. In response to such acontrol signal not shown, the processor 413 initiates reading the blockof quantization coefficients from the buffer 606 and the block attributeof the pertinent block from the FIFO memory 703, thereby starting theprocessing.

The video processor 413 includes a block data buffer 606 for storingtherein at least one block of quantization coefficients delivered from adecoder 604, a block attribute register 606 for storing therein blockattribute data 663 being processed, an inverse quantizer 607 forinversely quantizing quantization coefficients 653 outputted from thebuffer 606 according to a quantization parameter 655, an inverseorthogonal transform processor 608 for conducting an inverse orthogonaltransformation of coefficients 654 thus inversely quantized, an adder610 for adding an output from the processor 608 to a between-frameestimation value produced from a motion compensation processor 613, anda selector 611 for receiving as inputs thereto an output from the adder610 and an output from the processor 608 and conducting a selectingoperation according to an intra-frame encoding/between-frame encodingchange-over flag 661. For the intra-frame encoding, the output from theadder 610 is selected; whereas, for the between-frame encoding, theoutput from the processor 608 is selected. The obtained signal isoutputted therefrom as a decoded image 660. The video processor 413further includes a shared frame memory 614 for storing therein decodedimages of the related channels, a block address calculator 612 forcomputing read/write coordinates of the frame memory 614 according to areception channel signal 657 and a block number 656 of the block beingprocessed, and a motion compensation processor 613 operative accordingto a motion vector 659 for obtaining an estimated image between motioncompensation frames according to a previously decoded image stored inthe memory 614.

The operation of the processor 413 will now be described in more detailby reference to FIG. 8. The processor 413 is a section for processingand producing decoded images of several channels in a time sharingmanner. In the memory 641, there are independently prepared imagestorage areas 807a to 807c for respectively storing therein previousimages of the respective channels and image storage areas 808a to 808cfor respectively storing therein images currently being decoded for theassociated channels. A decoder section 802 conducts operations in theblock unit. In the decoder 802, there are arranged a processing sectionincluding the motion compensation processor 613, the inverse quantizer607, the inverse orthogonal transform processor 608, the block attributeregister 609, the adder 610, and the selector 611. For example, whenachieving processing for a channel A, the decoder 802 operates asfollows. While receiving one block of image data 801 via the channel A,the section 802 simultaneously reads a previous image 852 of thepertinent block from the storage area 807a of the memory 14, the area807 being disposed to keep therein an image decoded at a point of timeadvanced by one frame relative to the pertinent block. As a result, adecoded image 660 is written in the memory 614. In the memory 614, thestorage area of the previous image 852 are controlled for each channelaccording to a read area table 803. From the storages areas, anobjective storage area to read data therefrom is selected according tothe receiving channel information 657. A read block address calculator806 computes a read address including coordinates according to a blocknumber 851 being processed and an output from the table 803 so as tofeed the address to the memory 614. Similarly, the storage areas ofimages of the respective channels under the decoding operation arecontrolled according to a read area table A 804. An objective write areais determined according to the information 657. A write block addresscalculator 805 computes a write address including coordinates accordingto a block number 851 being processed and an output from the table 804and sends the obtained signal denoting the write address to the memory614. When the frame is completely processed for the channel, informationindicating the termination of the processing is written in the table 803so that the information in the related write area indicates a read areawhen processing a subsequent frame. Moreover, a new write area isreserved at the same time. Thanks to the configuration, the number ofreceiving channels can be arbitrarily specified within the limitation ofthe memory capacity. In addition, the respective channels are processedindependently of each other. Furthermore, for each channel, theresolution of the image communicated therethrough can be freelyspecified and can be dynamically changed. According to the embodiment,the shared frame memory 614 includes a single frame memory. In actualcases, however, the memory may be configured with several memory blocks,for example, in association with the write and read sides, therespective channels, and the like.

Returning again to FIG. 6, description will be given of an outline ofthe display frame memory 415. In accordance with the embodiment, themulti-site conference system includes four terminals at four points,respectively. Consequently, the memory 415 includes at least three sets(sides) of display frame memories X to Z (616 to 618), a display framememory write controller 615 for controlling write operations of decodedimages in the memories X to Z, a display frame memory read controller615 for controlling read operations of the memories X to Z, and adisplay sync generator 620 for generating a synchronizing signal fordisplay operations.

Referring now to FIG. 9, description will be given of a specificconfiguration and operation of the memory 415. This section includesthree sets of frame memories X to Z (616 to 618) and peripheral sectionsthereof. In the memory 415, images 660 asynchronously arriving thereatvia the respective channels are synchronized so as to produce an outputsignal 662. In the memories A to C, there are independently reservedareas to respectively store therein decoded images of the relatedchannels. For example, when images received via the channels A to C aredesired to be displayed on a screen, there are reserved storage areas906a to 906c respectively for the channels A to C in the memory X 616.Each of the storage areas is set to either one of four statusesincluding "display", "wait", "write", and "invalid" statuses. Thesestatuses of the storage areas are controlled according to a status tableA 904 and a status table B 905. In this embodiment, statuses relatedrespectively to write and read operations are kept in the tables A andB, respectively. As an alternative example, there may be used only onestatus table to collectively control write and read operations of thememory. In each memory, an area in which the image 660 received via eachchannel is written is controlled according to a write area table B 901.Based on the contents of the table B 901 and the block number 656,coordinates indicating an area of the memory in which the image 660 iswritten are created by a write address generator 902 to be sent to eachmemory. To write the decoded image 660 in one of three frames, theobjective frame is decided by the controller 615 according to the tableA 904. To display data, the read address generator creates a readaddress independently of operation on the write side according to asynchronizing signal generated by the generator 620 independentlydisposed in each terminal, thereby sending the address to each framememory. To read data from one of three memories, the objective memory isselected by the controller 619 according to the table B so as to outputthe data therefrom.

FIG. 10 is a signal timing chart showing operations of the memories 616to 618 of FIG. 9. Write operations are independently conducted for therespective channels. Description will be first given of a data receivingoperation via a channel A. When a new frame is received via the channelA at time t4, a write operation thereof is initiated in either one ofthe frame memories X and Y. The frame memory Z is being used to displaydata and hence is not employed for the data write operation. Namely,when the memory Z is in the display state, the frames arrived at thememory section 415 are sequentially written in the memories X, Y, X, Y,etc. in an alternate manner. When the memory Z is released from thedisplay state, the memory Z is set to the invalid state. A frame memoryin the wait state containing the latest frame for which the writeoperation has been terminated is then displayed and the write operationis achieved in the frame memory Z and the remaining available framememory. For example, when a fourth frame arrives thereat while a thirdframe already stored in the memory Z is being displayed, the writeoperation of the fourth frame is initiated in the memory X. When a fifthframe is subsequently arrived thereat, the write operation thereof iscommenced in the memory Y. In this situation, when the display state ofthe memory Z is terminated during the write operation of the memory X(t5), since the write operation of the latest frame is in process,namely, the fourth frame is being written in the memory X, the thirdframe of the memory Z is displayed again. Moreover, when the displaystate of the memory Z is finished during the write operation of thememory Y, since the write operation is already completed in the memoryX, the frame memory X is then displayed. When a subsequent framearrives, the write operation thereof is started in the memory Z. Inaddition, even in a case where the write operations are completedrespectively in the memories X and Y, when a new frame arrives thereatbefore termination of the display operation of the frame memory Z, thewrite operation of the frame is again initiated in the memory X. Withthe provision in which the write operations are independently carriedout for the respective channels and the synchronization timings areindependently established for the write and display operations, framesarriving at the memory section 415 at timings of the respective channelscan be advantageously displayed without any difficulty or trouble.

In the embodiment, an integer number of blocks are contained in eachpacket. However, when the packet length becomes to be equal to or lessthan the quantity of information equivalent to a block depending oncases or when a fixed value is stipulated as the packet length, thecoders 403a to 403d and decoders 404a to 404d are required to bemodified as follows.

For the coders 403a to 403d, the packetizer 511 of FIG. 5 need only bemodified according to required specification. For example, when handlingfixed-length packets, each time a fixed number of bits are received, ademarcation or delimiter header is added to information thus stored soas to output the obtained packet.

For the decoders 404a to 404d, the configuration of FIG. 7 is partiallymodified to configure a variation shown in FIG. 11. Modified portionsare as follows. There are added a register 1101 for storing thereinexcessive code bits of the variable-length decoder 701 and a buffer 1102for keeping therein excessive data of the buffer 606, the data having adata length less than the block length. In a case where the receivingchannel is not changed over, there is carried out the same operation asfor the construction of FIG. 7. However, there is added a function toconduct the following operation at reception of another packet.

After a packet is completely inputted, when a subsequent packet isreceived via another channel, the codes thus received are interpreted tothe possible extent to send the interpreted codes to the block databuffer or the block attribute FIFO memory. Thereafter, remaining codebits are moved from the decoder 701 to the storage area 1103a, 1103b, or1103c reserved for the pertinent channel in the register 1101. From thearea associated with the channel of the next packet, the code bits savedas the excessive data in the previous packet processing are readtherefrom so as to start again the decoding operation of the bits.

At the same time, while a block of data is being stored in the buffer606, when a packet is received via another channel, coefficient dataremaining in the buffer is saved in the area 1104a, 1104b, or 1104creserved for the pertinent channel in the buffer 1102. Simultaneously,from the area corresponding to the channel of the subsequent packet, theexcessive coefficient data of the previous packet is obtained. Inaddition thereto, there can be considered a variation in which blockdata buffers 606 are prepared as many as there are receiving channelssuch that a change-over operation is conducted according to the channelof a received packet.

Furthermore, as described above, the embodiment is implemented accordingto the algorithm conforming to the recommendation H. 261 for the videocoding standardization of the CCITT with modifications to cope with thepacket transmission and the multi-point connection. With a methodmaterialized according to the coding algorithm using correlation betweenframes in the block unit, for example, a vector quantization method,there can be obtained an advantageous effect similar to that of thepresent invention only by applying similar modifications according tothe present invention.

Moreover, in a case where an algorithm adopting correlation between aplurality of frames or fields is employed in the coding or decodingoperation, a necessary number of frame memories 502 of the coder and arequired number of shared frame memories 616 to 618 of the decoder needonly be disposed to achieve the processing.

A case where the functions above are implemented primarily by processingof software is also included in the scope of the present invention.

According to the present invention as described above, only one codingand decoding apparatus is required to be installed to conduct videocommunications to and from a plurality of points. Moreover, thecommunications with the respective sites can be independentlycontrolled, which leads to a remarkable advantageous effect inpractices.

While particular embodiments of the invention have been shown anddescribed, it will be obvious to those skilled in the art that variouschanges and modifications may be made without departing from the presentinvention in its broader aspects.

What is claimed is:
 1. A terminal for displaying a plurality of videoimages on a display screen, comprising:a receiver for receiving videodata packets from a plurality of video sources in a time sharing manner,each of said video data packets including coded video information forrepresenting a partial image contained within one of a plurality ofblocks constituting a video frame, attribute information for indicatingat least a location of a block corresponding thereto within the videoframe and identification information of the video source; a decoderconnected to said receiver for decoding the coded video informationextracted from each of said video data packets and for outputting apartial image corresponding to one of said blocks in the video frame; aset of frame memories for assembling partial images into video frames,each of said frame memories having a plurality of image areas located ina predetermined layout defined therein, a set of said image areas eachlocated at a same location within respective said frame memories beingassociated with a same video source with respect to each other; arecorder for writing each partial image derived from said decoder intoone of said image areas within one of said frame memories which isspecified in accordance with the identification information extractedfrom a video data packet related to the partial image and a utilizationstatus of said image areas; and a selector for selectively outputtingcontents of said frame memories on a display screen by changing oversaid frame memories depending upon a display position on the displayscreen thereby to present a plurality of video frames at a plurality ofimage areas on the display screen.
 2. A terminal as claimed in claim 1,further comprising: a memory for keeping therein status information foreach of said image areas, said recorder operating by referencing saidmemory so as to write said partial image into one of said set of imageareas which is specified by the identification information and asindicated in a status available for a write operation by said statusinformation, and said selector operating by referencing said memory soas to selectively output the contents of one of said image areas whichis indicated by said status information in an output waiting status orin an output execution status.
 3. A terminal as claimed in claim 1,wherein said decoder comprises a memory for storing said coded videoinformation for each video source until an amount of the coded videoinformation sufficient to produce one partial image is prepared in thedecoder.
 4. A terminal as claimed in claim 1, wherein said receivermeans is connected so as to receive said video data packets transmittedfrom at least one remote terminal through a communication network.
 5. Aterminal as claimed in claim 1, wherein said decoder comprises anextra-frame reproducer for producing partial images from first codedvideo information coded through an intra-frame encoding, and aninter-frame reproducer for producing partial images from second codedvideo information coded through an inter-frame encoding.
 6. A terminalfor displaying a plurality of video images on a display screencomprising:a packet supplier for sequentially supplying video datapackets each of which includes coded video information related to one ofa plurality of blocks constituting a video frame, attribute informationfor indicating a location of a block corresponding thereto within thevideo frame and identification information of a video source; a decoderconnected so as to receive said video data packets for decoding thecoded video information extracted from each of said video data packetsto output a partial image; a set of frame memories for assemblingpartial images into video frames; a memory for storing controlinformation for indicating a status of each of a plurality of videoareas defined within each of said frame memories in a predeterminedlayout and correlation between the video areas and video sources; arecorder for writing the partial image output from said decoder into oneof said video areas within one of said frame memories specified by thecontrol information in the memory and the identification informationextracted from the video data packet related to the partial image; and aselector for selectively outputting the contents of said frame memorieson a display screen by changing over said frame memories depending upona status of said video areas thereby to present a plurality of videoframes at a plurality of video areas on the display screen.
 7. Aterminal as claimed in claim 6, wherein said decoder comprises a buffermemory for storing said coded video information for each video sourceuntil an amount of the coded video information necessary to produce onepartial image is prepared in said decoder.
 8. A terminal as claimed inclaim 6, wherein said packet supplier for sequentially supplying saidvideo data packets is connected so as to receive video data packetstransmitted from at least one remote terminal through a communicationnetwork.
 9. A terminal as claimed in claim 6, wherein said decodercomprises an intra-frame reproducer for producing partial images fromfirst coded video information coded through an intra-frame encoding, andan inter-frame reproducer means for producing partial images from secondcoded video information coded through an inter-frame encoding.
 10. Amulti-point video conference system, comprising:a network; a pluralityof terminals interconnected via said network, with each terminalcomprising: a camera for capturing a video image; a coding and packetingarrangement for coding and packetizing said video images into video datapackets; a receiver for receiving video data packets from other ones ofsaid terminals in a time sharing manner, each of said video data packetsincluding coded video information for representing a partial imagecontained within one of a plurality of blocks constituting a videoframe, attribute information for indicating at least a location of ablock corresponding thereto within the video frame and identificationinformation of a terminal source; a decoder connected to said receivermeans for decoding the coded video information extracted from each ofsaid video data packets and for outputting a partial image correspondingto one of said blocks in the video frame; a set of frame memories forassembling partial images into video frames, each of said frame memorieshaving a plurality of window areas located in a predetermined layoutdefined therein, a set of said window areas each located at a samelocation within respective said frame memories being associated with asame terminal source with respect to each other; a recorder for writingeach partial image derived from said decoder into one of said windowareas within one of said frame memories which is specified in accordancewith the identification information extracted from a video data packetrelated to the partial image and a utilization status of said windowareas; and a selector for selectively outputting contents of said framememories on a display screen by changing over said frame memoriesdepending upon a display position on the display screen thereby topresent a plurality of video frames at a plurality of window areas onthe display screen.
 11. A video conference system as claimed in claim10, further comprising: a memory for keeping therein status informationfor each of said window areas, said recorder operating by referencingsaid memory so as to write said partial image into one of said set ofwindow areas which is specified by the identification information and asindicated in a status available for a write operation by said statusinformation, and said selector operating by referencing said memory soas to selectively output the contents of one of said window areas whichis indicated by said status information in an output waiting status orin an output execution status.
 12. A video conference system as claimedin claim 10, wherein said decoder comprises a memory for storing saidcoded video information for each terminal source until an amount of thecoded video information sufficient to produce one partial image isprepared in the decoder.
 13. A video conference system as claimed inclaim 10, further comprising a transmitter for transmitting said videodata packets to at least one remote terminal through said network.
 14. Avideo conference system as claimed in claim 10, wherein said decodercomprises an intra-frame reproducer for producing partial images fromfirst coded video information coded through an intra-frame encoding, andan inter-framed reproducer for producing partial images from secondcoded video information coded through an inter-frame encoding.
 15. Amulti-point video conference system, comprising:a network; a pluralityof terminals interconnected via said network, with each terminalcomprising: a camera for capturing a video image; a coding and packetingarrangement for coding and packetizing said video images into video datapackets; a sequencer for sequentially supplying said video data packetseach of which includes coded video information related to one of aplurality of blocks constituting a video frame, attribute informationfor indicating a location of a block corresponding thereto within thevideo frame and identification information of a terminal source; adecoder connected so as to receive said video data packets from otherones of said terminals for decoding the coded video informationextracted from each of said video data packets to output a partialimage; a set of frame memories for assembling partial images into videoframes; a memory for storing control information for indicating a statusof each of a plurality of video areas defined within each of said framememories in a predetermined layout and correlation between the videoareas and terminal sources; a recorder for writing the partial imageoutput from said decoder into one of said video areas within one of saidframe memories specified by the control information in the memory andthe identification information extracted from the video data packetrelated to the partial image; and a selector for selectively outputtingthe contents of said frame memories on a display screen by changing oversaid frame memories depending upon a status of said video areas therebyto present a plurality of video frames at a plurality of video areas onthe display screen.
 16. A terminal system as claimed in claim 15,wherein said decoder comprises a buffer memory for storing said codedvideo information for each video source until an amount of the codedvideo information necessary to produce one partial image is prepared insaid decoder.
 17. A terminal system as claimed in claim 15, wherein saidsequencer for sequentially supplying said video data packets isconnected so as to receive video data packets transmitted from at leastone remote terminal through said network.
 18. A terminal system asclaimed in claim 15, wherein said decoder comprises a means forproducing partial images from first coded video information codedthrough an intra-frame encoding, and a means for producing partialimages from second coded video information coded through an inter-frameencoding.
 19. A method of displaying a plurality of video images on adisplay screen, comprising the steps of:receiving video data packetsfrom a plurality of video sources in a time sharing manner, each of saidvideo data packets including coded video information for representing apartial image contained within one of a plurality of blocks constitutinga video frame, attribute information for indicating at least a locationof a block corresponding thereto within the video frame andidentification information of the video source; decoding the coded videoinformation extracted from each of said video data packets andoutputting a partial image corresponding to one of said blocks in thevideo frame; assembling partial images into video frames in a set offrame memories, each of said frame memories having a plurality of imageareas located in a predetermined layout defined therein, a set of saidimage areas each located at a same location within respective said framememories being associated with a same video source with respect to eachother; writing each partial image derived from said decoding step intoone of said image areas within one of said frame memories which isspecified in accordance with the identification information extractedfrom a video data packet related to the partial image and a utilizationstatus of said image areas; and selectively outputting contents of saidframe memories on a display screen by changing over said frame memoriesdepending upon a display position on the display screen thereby topresent a plurality of video frames at a plurality of image areas on thedisplay screen.
 20. A method as claimed in claim 19, comprising thefurther steps of: storing status information for each of said imageareas, referencing stored said status information so as to write saidpartial image into one of said set of image areas which is specified bythe identification information and as indicated in a status availablefor a write operation by said status information, wherein saidselectively outputting step references stored said status information soas to selectively output the contents of one of said image areas whichis indicated by said status information in an output waiting status orin an output execution status.
 21. A method as claimed in claim 19,wherein said decoding step includes the substep of storing said codedvideo information for each video source until an amount of the codedvideo information sufficient to produce one partial image is prepared.22. A method as claimed in claim 19, wherein in said receiving step,said video data packets are received after being transmitted from atleast one remote terminal through a communication network.
 23. A methodas claimed in claim 19, wherein said decoding step includes alternativesubsteps of producing partial images from first coded video informationcoded through an intra-frame encoding, and producing partial images fromsecond coded video information coded through an inter-frame encoding.24. A method of displaying a plurality of video images on a displayscreen, comprising the steps of:sequentially supplying video datapackets each of which includes coded video information related to one ofa plurality of blocks constituting a video frame, attribute informationfor indicating a location of a block corresponding thereto within thevideo frame and identification information of a video source; receivingsaid video data packets for decoding the coded video informationextracted from each of said video data packets to output a partialimage; assembling partial images into video frames using a set of framememories; storing control information for indicating a status of each ofa plurality of video areas defined within each of said frame memories ina predetermined layout and correlation between the video areas and videosources; writing the partial image output from said receiving step intoone of video areas within one of said frame memories specified by thecontrol information and the identification information extracted fromthe video data packet related to the partial image; and selectivelyoutputting the contents of said frame memories on a display screen bychanging over said frame memories depending upon a status of said videoareas thereby to present a plurality of video frames at a plurality ofvideo areas on the display screen.
 25. A method as claimed in claim 24,wherein said receiving step includes storing said coded videoinformation for each video source in a buffer memory until an amount ofthe coded video information necessary to produce one partial image isprepared.
 26. A method as claimed in claim 24, wherein in saidsequentially supplying step, said video data packets are transmittedfrom at least one remote terminal through a communication network.
 27. Amethod as claimed in claim 24, wherein said receiving step includesalternative substeps of producing partial images from first coded videoinformation coded through an intra-frame encoding, and producing partialimages from second coded video information coded through an inter-frameencoding.